closes #21

This PR introduces a standalone data quality monitoring module designed to ensure the reliability, integrity, and timeliness of data feeds powering trading decisions. The module provides real-time validation, anomaly detection, monitoring, and reporting to reduce risks associated with corrupted or delayed data.

Key Features & Tasks Completed

• Data Validation Rules & Schemas: Enforced input validation to block corrupted or malformed data.
• Real-Time Anomaly Detection: Implemented algorithms to flag unusual patterns within minutes.
• Freshness Monitoring: Detects stale or delayed data feeds, triggering alerts on delays greater than 15 minutes.
• Data Completeness Checks: Identifies gaps and ensures consistent coverage across feeds.
• Statistical Outlier Detection: Uses robust statistical models with <5% false positive rate.
• Automated Data Correction & Flagging: Automatically resolves 80%+ of common issues and flags unresolved cases for manual review.
• Dashboards & Alerts: Provides real-time visibility into data health, anomalies, and overall system reliability.
• Data Lineage Tracking: Maintains visibility into data flow, transformations, and dependencies.
• Source Reliability Scoring: Generates ongoing reliability scores for each upstream data provider.
• Automated Reporting: Produces periodic reports summarizing data quality KPIs, issues, and trends.

Acceptance Criteria

• Detects anomalies within 5 minutes of occurrence.
• Prevents 99%+ of corrupted data from entering the system.
• Alerts when data delays exceed 15 minutes.
• Outlier detection achieves a <5% false positive rate.
• Dashboard reflects real-time data quality and system health.
• Automated corrections resolve 80%+ of recurring issues.
• Reliability scoring provides accurate data source evaluation.

Impact

This module significantly enhances the trustworthiness of trading data, reduces manual intervention, and strengthens overall decision-making reliability. It creates a foundation for long-term scalability, auditability, and risk reduction in the trading platform.